OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association

Many image-based perception tasks can be formulated as detecting, associating and tracking semantic keypoints, e.g. , human body pose estimation tracking. In this work, we present a general framework that jointly detects forms spatio-temporal keypoint associations in single stage, making the first real-time detection algorithm. We generic neural network architecture uses Composite Fields to detect construct xmlns:xlink="http://www.w3.org/1999/xlink">spatio-temporal pose which is single, connected graph whose nodes are keypoints ( xmlns:xlink="http://www.w3.org/1999/xlink">e.g ., person’s joints) multiple frames. For temporal associations, introduce Temporal Association Field (TCAF) requires an extended training method beyond previous Fields. Our experiments show competitive accuracy while being order of magnitude faster on publicly available datasets such COCO, CrowdPose PoseTrack 2017 2018 datasets. also our generalizes any class car animal parts provide holistic well suited for urban mobility self-driving cars delivery robots.